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Abstract: In this paper, an evaluation of web sites regarded like projects is discussed. We give 
an overview of the Web Assessment Index (WAI), by presenting a web sites of Romanian 
Faculties of Economics case study. The WAI contains five categories: accessibility, access speed, 
navigability, content and reliability. We analyzed and presented a detailed report of the results 
coming out from this study. This study is important to understand the issues on which the 
faculties web sites are confronting. 
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1. Introduction 

Projects are temporary endeavors undertaken to create unique products or 
services. A project has settled start date, end date, milestones, scope and objectives; 
produces unique results and is characterized by a certain approach and a 
progressive elaboration 

Nowadays, educational Web sites are considered as projects developed by 
Universities or other entities for their presentation, recruiting students and supplying 
different educational services. This is the reason why we have decided to evaluate 
the academic web sites of the most important Romanian Universities. For this 
assessment we need a flexible, engineering-based methodology that allows us to 
measure the degree of goal achievement of educational projects through the 
academic web sites. 
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In order to evaluate this kind of projects it is necessary to analyze them 
from different aspects, such as: the way they are managed, the costs/benefits report, 
the upgrade possibility, the potential risks, but most of all, in our opinion, we should 
pay a special attention to the quality feature. 

Along with the web technology development, the education institutes have 
chosen for their presence on the Internet, by means of web sites. These ones are 
developed to offer general information about the universities, to support the 
education process or to carry on the distance education process over the Internet, 
exclusively. 

Information technology has been successfully combined with the learning 
process, resulting the e-learning technology, which provides new learning 
opportunities with less restriction on time and space. Academic e-learning initiatives 
aim at designing, implementing and introducing an e-learning system in a higher 
education institution [Mendling, 2004]. 

The purpose of this paper is to develop a web site assessment index that 
can be employed to estimate the current usage of the Internet by the most famous 
Romanian universities for their educational process support and at the same time the 
satisfaction of specific requirements from user's viewpoint. Web sites, as essential 
component in modern academic educational projects, necessitate continuous 
evaluation for supervising the fulfillment of learning objectives stipulated in these 
projects. 

The web site quality assessment is necessary because the web is an 
increasingly important source of information and there is no way to control the 
quality of published content. 


2. Web sites assessment methods 

In order to achieve our goals, we have established the main necessary 
indicators for the web sites quality evaluation in education domain. 

A review of the recent literature on web site reveals a lot of criteria for the 
web sites assessment. These elements are used in a quantitative evaluation, 
comparison, and ranking process [Polla, 2007; Shahin, 2004]. 

One of the most common method was proposed by Olsina et al. 1999 and 
can be considered as one of the main approaches. This method is the web site 
Quality Evaluation Method (QEM). There are four main factors analyzed in Olsina's 
study: functionality, usability, efficiency and site reliability [Olsina et al., 1999]. 
Currently, the results of the web sites evaluation are very subjective; thus, site 
evaluators should be given precise guidelines to rate every factor. In order to avoid 
this subjectivity problem, a Web Assessment Index (WAI) can be used. According to 
Evans and King, this index represents a web assessment tool and must have five 
main components: categories, factors, weights, ratings and total score [Evans and 
King, 1999]. 

In this paper, we have employed this assessment tool to compare the 
Internet usage from the five largest Romanian universities of the most famous 
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university centers of the province: la§i, Cluj-Napoca, Timi§oara, Constanta, Bra§ov. 
Because of paper limited space, we have been reported only on Faculties of 
Economic Sciences of these universities (we mention that the Academy of Economic 
Sciences of Bucharest is constituted of 1 1 faculties and colleges, so we have not 
been reported on this one). The list below presents the faculties web sites of the first 
five Romanian universities. 

Table 1 . Classification of Romanian universities based on stuff's scientific results 


with international relevance in 2006 


University Centre 

Faculty of Economic Sciences web site 

"Al. 1. Cuza" University of lassy 

http://www.feaa.uaic.ro 

"Babes-Bolyai" University of Cluj-Napoca 

http://www.econ.ubbcluj.ro 

"West" University of Timisoara 
"Ovidius" University of Constanta 
"Transilvania" University of Brasov 

http://www.fse.uvt.ro 

http://www.univ-ovidius.ro/stec 

http://econ.unitbv.ro/ed/default.aspx 


Source: Ad Astra Association, Topul universitatilor din Romania - Clasamentul general - 2006 


(http://www.ad-astra.ro/universitati/universities.php?year=2006) 

The WAI proposed by Miranda et al. (2006) employs four categories that 
are critical to web site effectiveness. These four categories are: accessibility, speed, 
navigability, and site content [Miranda, Cortes and Barriuso, 2006). 

In order to complete our empirical study, we have extended Miranda's 
proposed index at five categories: accessibility, speed, navigability, content, 
reliability. In our opinion, these five categories are essential for web sites 
assessment. For each category, we have chosen some factors which reflect the most 
important components and features of web sites from the user's viewpoint. 

The first category in the WAI is accessibility. This criterion is an influence 
factor of the site quality; that is, the identification and access facility (from user's 
viewpoint) is in direct proportion with the quality. The web accessibility can be 
evaluated analyzing two factors: link popularity and search engines presence. 

The accessibility of a web site is increased by the search engines ranking. 
In order to establish the search engines ranking of the five web sites, we have 
chosen Google search engine, this one being the most frequently used in our 
country and considered the best in the world. The higher search engine ranking is, 
the higher degree of accessibility is. 

Link popularity is the number of external links to the assessed web site. 
This has been measured using Google for counting the link occurrences of the 
reviewed sites in other different sites. 

A significant issue for sites visitors is access time. In our paper, we analyzed 
the access speed from time viewpoint. The access speed was measured using a 
chronometer [Miranda, Cortes and Barriuso, 2006]. In order to eliminate any 
disagreements in access speeds measuring of the five web sites, we carried the 
access speed tests at the same hour with the same computer (Intel PIV 2,66 GHz, 
512MB of RAM, 64MB graphic card) which has a 1MB optic fiber connectivity. We 
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used Internet Explorer 6.0, Opera and Mozilla Firefox 2.0 browsers. The access 
speed was calculated as the average between the three time values, reported at the 
number of pages for each web site. This procedure was repeated on consecutive 
days so that the averaged speed of access to be more representative. 

The navigability is a very important factor from user's viewpoint because it 
refers to user's ability to reach the desired location in very short time. In this 
category, we analyzed: the presence of a permanent menu which allows a fast 

access to the different sections (at least the presence of a Home button) and the 
presence of a site map button which allows us to see the structure of the web site. 
Also, a search function is a very important factor. 

The content is the most important category, in our opinion, because the 
presented information is the attraction key factor. That is why we have focused on 
the assessment of this category. In order to evaluate the content of the five web 
sites, we decomposed the content in attributes, in lower levels of abstraction, so as 
to be able to effectuate the measurement. Thus, we considered the following levels 
of content: informational level, services level, scientific research level, 

communicational level and miscellaneous level. 

The first level - information - has a general descriptive character therefore 
we have followed the decomposition for assessment process. We were interested in 
the presence of: general faculty information (departments, management); entrance, 
educational forms; university degree; syllabus, timetable; announcements; financial 
information. 

The second level takes into account the services available on the web, 
especially in student's self-interest: digital library (download courses, on-line reading 
and other on-line resources); on-line support services (view of the results 
centralization, using a login form); scholarship; symposiums, magazines. 

The scientific research level addresses to anyone who wants to participate 
at conferences, to publish scientific articles or to gain scholarships. Generally, the 
academic personnel are the target of this level. 

The communicational level refers especially to contact information, such as: 
headquarters address; telephone/fax; e-mail/web; form-based feedback. 

Reliability is an important category both for visitors and for faculties 
because it highly affects their site assessment. This category is evaluated using two 
factors: link errors and miscellaneous errors. Link errors can be hyperlinks which do 
not work from different reasons, invalid links that are not correctly implemented and 
unimplemented links that are not implemented at all (they are measured through the 
number of links that are invalid, respective unimplemented reported to the total 
number of links). 

Miscellaneous errors consist of three factors: deficiencies or absent features 
due to different browsers, deficiencies or unexpected results (e.g. non-trapped 
search errors, frame problems, etc.) independent of browsers and inactive nodes 
(unexpectedly under construction or dead-end Web nodes). 
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3. Data analysis 

In order to evaluate these web sites, for each category was assigned a 
weight, established on according to importance degree (from a total of 100 points). 
Every factor in every category has been rated (from the total of category). 


Table 2. Web Assessment Index with the five category and their weights 


CATEGORIES 

Weights 

CATEGORIES 

Weights 

Accessibility 

10 

Navigability 

10 

Presence in search engine 

5 

Permanent menu 

4 

Link popularity 

5 

Site map 

3 



Search function 

3 

Speed 

10 

Reliability 

10 

Access speed 

10 

Link errors 

6 



Miscellaneous errors 

4 

Content 

60 


Informational level 

15 

Services level 

15 

General faculty information 

3 

Digital library 

6 

Entrance, educational forms 

3 

Marks centralization 

4 

University degree 

3 

Scholarship 

3 

Syllabus, timetable 

3 

Symposium 

1 

Financial information 

3 

Magazines 

1 

Scientific research level 

15 

Communicational level 

15 

Conferences, symposiums 

6 

Address, telephone 

4 

Journals, magazines 

6 

E-mail 

4 

Scholarship 

3 

Form-based feedback 

7 

TOTAL 

100 


For the first category, we have considered the Position attribute, which 
denotes the range number of the main site of the faculty (index) within results of 
Google expression search. The expression represents the faculty of each University, 
followed by the city name, such as: "facultatea de §tiinte economice cluj-napoca" or 
"faculty of economic sciences cluj-napoca". The criterion function for determining 
the rating of the presence in search engine factor, is [Olsina, Rossi, 2000): 

case Position of 

0: Rt = 0; 1: Rt = 5; 

2: Rt = 4; 3: Rt = 3; 

otherwise if (Position < =10) then Rt = 2 else Rt = 1. 


For the evaluation of link popularity, we have employed an advanced 
search option of Google search engine. For every web site we have counted sites 
that contain links to the faculty web site, Lk, (1<i<6), and we have calculated an 
average variable, Av. In order to establish the link popularity for each faculty site, we 
compared the Lk with the Av variable. 

Av = (Lkj + Lk 2 + Lki + Lk 4 + Lk s + L k / 6; 
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Rating (Rt) 



Figure 1 . Popularity link assessment 


If the web site structure includes a functional permanent menu available in 
every location which allows us to navigate in the whole web site, the permanent 
menu factor gets 4 points rating (by permanent menu we understand the principal 
menu, which is present by his buttons in every section of the web site, in top of the 
page). Otherwise, it gets 0 points rating (if the menu doesn't exist) or 2 points rating 
(if it exists but is not functional). 

If there is a tree permanent menu, the factor gets only 3 points if it is 
functional, 1 point if it is not functional or 0 points if it is not permanent. 

If there is no permanent menu, but there is a permanent Home button, the 
points are 3, 1 and 0, corresponding the three cases (exists and is functional, exists 
but is not functional, doesn't exist). In the same manner, we have rated the site map 
and intern search function with 3 points, 1 point or 0 points (corresponding the 
three cases presented above). 

The content's factors ponderability has been made depending on some 
aspects, as follows: 

• for the first level - we have considered some aspects, regarding 
information, such as: quality, amount, enframming within page, last 
update; 

• for the second level - we have focused on student services offer, 
especially on digital library service. We have rated these factors depending 
on the service complexity and functionality; 

• for the scientific research level - we have analyzed the possibility of 
research activity development for the faculty's personnel and for others 
academics. We have rated this factor depending on amount of conferences 
symposiums, journals, magazines (like Acta Universitatis). We also consider 
very important (maybe more important than the amount characteristic) 
their bench mark (if there are international or national rated, if they are 
recognized by CNCSIS etc.). 

• the feedback level (or the communicational level) implies the existence 
of contact information, such as address, telephone, e-mail. We have also 
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looked for the feed-back availability, meaning the presence of a form 
which allows the user to ask any questions about the faculty. 


Link errors (£/) have been measured through the number of links that are 
invalid, respective unimplemented (Li) reported to the total number of links (77). In 
order to establish the points for this factor (Epi), we have multiplied this coefficient El 
by 100 and we have truncated it to obtain natural numbers, named link errors index 
(Eli), which have been compare using a scale from 0 to 10 (we have chosen the 
upper limit at 1 0, because the bigger natural number was 9). 

El = Li / Tl; 

Eli = Trunc (El * 100); 

if Eli = 0 then Epi = 6 
else if Eli < = 2 then Epi = 5 

else if Eli < = 4 then Epi = 4 

else if Eli < = 6 then Epi = 3 

else if Eli < = 8 then Epi = 2 

else if Eli <=10 then Epi = 1 . 


Miscellaneous errors (Em) have been measured through the number of 
specified attributes (Sa) considering a scale. In order to find out miscellaneous errors, 
we tested the five web sites using two browsers: Internet Explorer and Opera. Then 
we have calculated an average variable, Avm (the average of the Sa variables). The 
factor was rated with points (Mp) from 0 to 4, comparing with the 0, Avm/ 2 and Avm 
values. 


Em 


Mp 


0 

•- 


e 


Avm/2 

• — 

2 

— )(- 


Avm 


3 1 

Figure 2. Miscellaneous errors assessment 




After the settlement of these assessment criteria, we have started the 
evaluation of the five web sites - projects based on empirical case study. Using the 
above criteria, we calculated for each factor and each category, the rating and the 
weight, respectively. We present a table containing the five categories and their 
weights: 


Table 3. Romanian Faculties of Economics Web Sites Assessment Index 


Categories / 
Universities 

Al.I.Cuza 

Iasi 

Babes-Bolyai 

Cluj-Napoca 

West University 
Timisoara 

Ovidius 

Constanta 

Transilvania 

Brasov 

Accessibility 

6 

10 

6 

2 

6 

Presence in 
search engines 

5 

5 

5 

1 

5 

Popularity 

1 

5 

1 

1 

1 

Speed 

10 

10 

10 

10 

10 

Access speed 

10 

10 

10 

10 

10 
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Categories / 
Universities 

Al.I.Cuza 

Iasi 

Babes-Bolyai 

Cluj-Napoca 

West University 
Timisoara 

Ovidius 

Constanta 

Transilvania 

Brasov 

Navigability 

10 

4 

7 

7 

3 

Permanent menu 

4 

4 

4 

4 

3 

Site map 

3 

0 

0 

0 

0 

Search function 

3 

0 

3 

3 

0 

Content 

51 

36 

32 

28 

32 

Informational 

level 

14 

13 

9 

11 

12 

Services level 

9 

6 

3 

8 

7 

Scientific level 

13 

9 

12 

3 

9 

Feedback level 

15 

8 

8 

6 

4 

Reliability 

4 

9 

1 

10 

9 

Link errors 

4 

5 

1 

6 

6 

Miscellaneous 

0 

4 

0 

4 

3 

errors 






TOTAL 

81 

69 

56 

57 

60 


4. Results analysis 

Although we have synthesized the results of our work in a graphic which 
presents the best web sites according to the WAI, we are not interested on identifying 
the best web sites, but on analyze how each faculty web site is compared to related 
sites and how can they be improved for their educational goals achievement. 

According to the WAI, the best web sites correspond to the Faculties of 
Economic Sciences of "Al. I. Cuza" University of Iasi and "Babes-Bolyai" University of 
Cluj-Napoca. 


The Web Assessment Index values 


"Transilvania" of Bra§ov 
"Ovidius" of Constanta 
"West" University of Timi§oara 
"Babe§-Bolyai" of Cluj-Napoca 
"Al. I. Cuza" of la§i 



□ The Web Assessment Index values 


Figure 3. The WAI values for the five Romanian Faculties of Economic Sciences 
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From the accessibility viewpoint, it's obvious that the faculties of Iasi, Cluj- 
Napoca, Timisoara and Brasov hold the best presence in search engines, meaning 
their web sites are the first outputs of Google search engine. Related to the 
popularity, things are changed, meaning only the faculty of Cluj-Napoca has a 
popularity of 606 results for its web site (www.econ.ubbcluj.ro) while the others have 
a popularity of 2 results for their web sites. 


The accessibility category 

100 
80 
60 
40 
20 
0 

Presence in search engines Popularity 

□ "Al. I. Cuza" of la§i ■ "Babe§-Bolyai" of Cluj-Napoca 

□ "West" University ofTimi§oara □"Ovidius" of Constanta 

□ "Transilvania" of Bra§ov 

Figure 4. The two factors of the accessibility category 



Regarding the five web sites access speed, we consider that all the web sites 
have an optimum access time reported to their number of pages and there are no 
lags in their access. So they all get a 1 00% percent, meaning they all get 1 0 points in 
the web assessment index. 

Excepting the Faculty of Economic Sciences of Brasov which has a tree 
menu, difficult to use, the other faculties web sites hold a permanent menu tree, in 
top of pages, which is easy to use, permitting the navigability in every section of the 
web site. Only one of the five reviewed web sites has a site map, and this is Faculty 
of "Al. I. Cuza" University of Iasi. The site map is a very important element of the 
web structure, because it allows us to take a look at the entire site, simplifying the 
localization of the sections we are interested in. The search function is very important 
for the site, because it performs an intern search, within the web site. Only three of 
the five web site analyzed, own a keyword search function. 

If we appertain to the entire category, we can notice that, considering the 
navigability, on the first place is the Faculty of Economic Sciences of Iasi, with a 
100% percent, followed by the faculties of Timisoara and Constanta with a 70% 
percent, faculty of Cluj-Napoca with a 40% percent and the faculty of Brasov with a 
30% percent. Even if the faculty of Cluj-Napoca is the second best web site of the 
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five sites analyzed, it has a lot of failings from navigability viewpoint and navigability 
elements could improve a lot this web site. 


The navigability category 

1 00 n 

75- 
50- 
25 
0 

Permanent menu Site map Search function 


□ "Al. I. Cuza" of la§i □ "Babe§-Bolyai" of Cluj-Napoca 

□ "West" University of Timi§oara □ "Ovidius" of Constanta 

□ "Transilvania" of Bra§ov 

Figure 5. The factors of navigability category 

The content category is the most important aspect in our study. As we can 
see, there is a various graphic for content category, meaning the five web sites are 
very different from this point of view. That is, their services are still deficient. At this 
category, the Faculty of Iasi offers the best services for their students and personnel 
with a 78,46% percent, followed by Cluj-Napoca with a 55,38% percent, Timisoara 
and Brasov with a 49,23% percent and Constanta with a 43,07% percent. The 
informational level is the only level where the all five web sites have a percent bigger 
then 50%, while the services level (services for students) is the level where the higher 
percent, 60%, is attained by Iasi and the lowest percent, 20%, is attained by 
Timisoara. 

After this category study, we consider that this five web sites are designed 
for presentation, more than for education support. In our opinion, these web sites 
should be improved, so that they can be a real support for the students, namely it 
should be implemented a digital library and they should offer the students the 
possibility to find out information about their results centralization, based on an 
account with user and password for login on their personal page. 

At scientific research level, the faculty of Iasi holds a 86% percent, followed 
by Timisoara with a 80% percent of scientific research information. 

It is very important for a web site structure to contain a feedback 
establishment. Generally, the web sites hold a Contact button which allows to find 
out the address information, telephone (general information contact), but also there 
is a form that enables the feedback establishment by asking questions via the form. 
From the five web sites analyzed, only the faculty of Iasi settles a feedback with their 
users, the other faculties offering only general information contact. 
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□ "Al. I. Cuza" of la§i ■ "Babe§-Bolyai" of Cluj-Napoca 

□ "West" University of Timisoara □"Ovidius" of Constanta 

□ "Transilvania" of Bra§ov 


Figure 6. The levels of the content category 


Regarding reliability, the most reliable web sites are the sites of the 
faculties of Constanta with a 100% percent, Cluj-Napoca and Brasov with a 90% 
percent compared with the site of Iasi with an only 40% percent and Timisoara with 
a 10% percent. The reliability is a very important aspect of a web site, meaning that 
no matter how complex the site is, it should not contain any link errors or any 
miscellaneous errors. 


100 


80 


60 

40 

20 

0 


□ "Al. I. Cuza" of la§i o "Babe§-Bolyai" of Cluj-Napoca 

□ "West" University of Timi§oara □ "Ovidius" of Constanta 

□ "Transilvania" of Bra§ov 

Figure 7. The analyzed factors of the reliability category 
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5. Conclusions and discussions 


Although the Faculty of Economic Sciences from lassy city has a best web 
sites (according to the WAI), its web site can be much improved regarding the 
reliability and the popularity of the web site. From the five web sites analyzed it 
offers the best students services and the higher possibility for scientific research. 

The web site of the Faculty of Economic Sciences from Cluj-Napoca city, 
could be improved by adding a site map and a search function and by diversifying 
the services offered to the students. 

The web site of the Faculty of Economic Sciences of Timisoara, has many 
link errors, which means that it is not functional enough. It could be improved by 
working at the reliability, at popularity and also diversifying the set of the services. 

The Web site of the Faculty of Economic Sciences from Constanta city has a 
very bad presence in the search engines and popularity, although it has highest 
reliability of the five sites reviewed. It necessitates diversifying of the services offered 
and could be improved by adding a site map. 

The last site analyzed is the web site of the Faculty of Economic Sciences 
from Brasov city. In our opinion, the tree menu should be changed by another easier 
to use (a menu containing submenus and options), it could be added a site map and 
a search function, but also it is recommended to diversify the services for the 
students. 

We conclude that Romanian universities have partly accomplished the 
objectives of supporting their educational project by means of the academic web 
sites reviewed in this paper and they need improvement where the WAI indicator has 
recorded poor results or absent characteristics. 

Although many firms and institutes have a web site which is intended to be 
their image, in many cases this web sites does not complete all the characteristics of 
the web site assessment. Many times, the only concern is the web design aspect, 
which is not enough for a site. There are more aspects on a web site than web 
design, aspects on which we have debated above. 
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